AI Lessons
Lesson 13: AI and Audio - How does AI create Voices and Music?

Purpose: How AI generates speech and music by learning patterns in sound, not by understanding language, emotion, or music.

No. of Classes

2 - (Time : 1 hour 30 minutes each, Laptops/desktops : 10, Students strength : 15 to 20).

Materials Required

Laptop / Desktop with Internet connections / Wi-Fi.

Headphones or speakers, and a microphone for the sound classifier exercise.

Curriculum Content

1. AI Literacy

  • 1.3 Using AI tools

2. AI and ML Concepts - A Simplified Overview

  • 2.2 Generative AI fundamentals
  • 2.4 Digital representation of media
  • 2.7 Model accuracy and confidence

4. Working with Pre-trained AI Models

  • 4.6 Audio classification
  • 4.9 Detection failure and edge cases
  • 4.10 Speech synthesis
Prior knowledge
  • Browsers
  • Basics of prompting
  • AI generates by predicting the next unit from patterns
Exercises

Exercise (1)


  • Explore how sound is created and represented as sound waves. Observe how a computer captures and stores sound as digital data.

Exercise (2)


  • Use an AI sound classifier to identify different sounds. Observe how it classifies sounds, what confidence score it gives, and where it gets the answer wrong.
  • Click here to open the MediaPipe audio classifier.

Exercise (3)


  • Use an AI sound generation tool to create sound effects from text prompts. Observe how different prompts produce different sounds.
  • Click here to open the ElevenLabs sound effects tool.

Exercise (4)


  • Convert text into speech and generate music with AI tools. Compare the two outputs and discuss how AI creates different kinds of audio from text.
  • Click here to open TTSMaker for text to speech, and click here for Suno Music.

Solutions


Teacher's Instruction:
  1. Check the availability of the AI tools used in this lesson and read the instruction documents for all the exercises before the class. Some of the sound tools ask for a sign-in and have a daily limit, so try them beforehand.
  2. Sound is data, in the same way that images and text are data. It is turned into numbers before AI can do anything with it.
  3. Explain what a sound wave is showing - how loud, and how fast it wobbles. Louder and higher sounds look different on the screen, and that difference is what gets stored as numbers.
  4. An image is a grid of numbers taken all at once, but sound is numbers taken one after another over time. That is why patterns in audio are patterns of change rather than patterns of shape.
  5. In Exercise 2 draw attention to the confidence score, not only the label. Try a sound the classifier has probably never been taught, and let the students see it guess confidently and wrongly.
  6. In Exercises 3 and 4 the tool predicts what the sound should be like from the words in the prompt, exactly as the image and video tools did earlier in the course. It is the same idea applied to a different kind of media.
  7. Points to Ponder:
    • If AI can write a tune in seconds, what is left that a musician can do that AI cannot?
    • Audio is represented as numbers, so your words become numbers. If two people in this room say the same word, are the numbers the same or different? What makes them different?